Overview
- Week 8: The Linear Model
- TODAY: The Linear Model with multiple predictors
- [3 weeks for Spring vacation]
- Week 10: Effect sizes
- Week 11: Consolidation
Linear Models (LM): recap
- Trying to make predictions about the world
- Capture the relationship between predictor & outcome
- Linear model (line) is described by an intercept (b0) and a slope (b1)
\[Outcome = b_0 + b_1\times Predictor_1 + \varepsilon\]
- Use the model to see:
- R2: proportion of variance in outcome that model explains
- t-statistic and associated p-value: is b1 different from 0?
- Direction of relationship between predictor & outcome
Today’s Topics
- How good is our model?
- Our model vs the mean
- Error in the model
- Throwing more predictors into the mix!
- Comparing hierarchical models
- Comparing predictors in a model
- Bringing it all together: what can our model tell us?
How good is our model?
- A good linear model (LM):
- Explains a lot about our outcome
- Captures more than using the simplest model possible
- Doesn’t contain much error in prediction
Keep is simple, stupid: The mean model

- We want a LM that explains more than the simplest model possible
- The simplest model is the mean…
- Mean chocolate bars eaten per week is 3.27
- Predict how much chocolate your neighbour eats a week…
- Let’s plot this
- Chocolate eaten is our outcome variable
- Not a great model!

- Let’s add a predictor
- Error in model
- between prediction (using mean) and observed data

- Let’s add a predictor
- Error in model
- between prediction (using mean) and observed data

We can do better, probably…
- Let’s construct a LM between chocolate liking and eating
- The line explaining our data with the least error
- Compare to the line representing our mean model
- Shows the improvement in prediction from LM (vs mean)

- Let’s construct a LM between chocolate liking and eating
- The line explaining our data with the least error
- Compare to the line representing our mean model
- Shows the improvement in prediction from fitting a LM

- Larger difference = greater improvement!
- We want a LM that explains more than the mean model

Interim summary
- Is our LM is better than the simplest model possible?
- The mean model is the simplest model
- We want a LM that explains more than the mean model
- But…a LM better than the mean still contains error
- How much error is okay?
- Does the model explain more than it doesn’t explain?
- How well does the LM predict the outcome?
Error in the model
- No model will fit the data perfectly
- Fit a line to best capture relationships between variables
- Want the least error possible
- Compare predicted and observed data points
Bringing it all together: the F-statistic
\[F =\frac{what\ the model\ can\ explain}{what\ the\ model\ cannot\ explain }=\frac{signal}{noise}\]
We want signal to be as big as possible
We want noise to be as small as possible
A ratio of variance explained relative to varience unexplained
Ratio > 1 means our model can explain more than it cannot explain
Associated p-value of how likely we are to find a F-statistic as large as the observed if the null hypothesis is true
Why you gotta complicate things?

- Multiple predictors are not much more complicated!
- We build models to predict what is happening in the world
- Simple explainations for complex relationships?
- Multiple predictors = greater explanatory power
How multiple predictors (don’t really) change the LM equation
\[\begin{aligned}Outcome &= Model + Error\\
Y&=b_0 + b_1\times Predictor_1 + \varepsilon\\
&=b_0 + b_1\times Predictor_1 + b_2\times Predictor_2 + \varepsilon\end{aligned}\]
- Y: outcome
- b0: value of outcome when predictors are 0 (the intercept)
- b1: change in outcome associated with a unit change in predictor 1
- b2: change in outcome associated with a unit change in predictor 2
How multiple predictors (don’t really) change the LM equation
- One predictor LM = regression line
- Two+ predictor LM = regression plane
Puppies, puppies everywhere!
The outcome: 
Predictor 1: 
Predictor 2: 
\[happiness\ =\ b_0 + b_1\times puppies\ +\ b_2\times fluffy + \varepsilon\]
- Y: happiness
- b0: value of happiness when puppies and fluffy are 0 (the intercept)
- b1: change in happiness associated with a unit change in puppies when fluffy is 0
- b2: change in happiness associated with a unit change in fluffy when puppies is 0
Model fit: F-statistic
- The F-statistic is 52.38
- The associated p-value is < .001
- Adding predictors significantly improved model fit
- Explained significantly more variance than could not explain
Model fit: R2 value
- The R2 is 0.74
- 74% of the variance in happiness was explained by puppies and fluffy ratings
- This is just in our observed data!
- Our adjusted R2 value was 0.73
- If we used the same model with the population, we should be able to explain 73% of the variance in happiness
Interim summary, mark II
- The LM can be expanded to include additional predictors
- The model is still described by an intercept (b0)
- It now includes slopes (bs) for each predictor
- This creates a regression plane instead of a line
- We can assess how good this model is
- F-ratio and associated p-value
- R2
- What if we want to compare two linear models?
Comparing linear models
- We can compare models with different numbers of predictors
- See which model better captures our outcome
- The models must be ‘hierarchical’
- 2nd model has the same predictors as the 1st model plus extra
- 3rd model has the same predictors as the 2nd model plus extra
Even more puppies!
Predictor 3: 